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A steganographic method is disclosed to embed an invisible watermark into an image. It 
can be used for copyright protection, content authentication or content annotation. The technique 
is mainly based on K-L transform. Firstly a block and cluster step (106) and cluster selection step 
(108) are performed to enhance the optimization of K-L transform (1 10) for a given image. Then 
a watermark is embedded (114) into the selected eigen-clusters. ECC (Error Correction Code) can 
be employed to reduce the embedded code error rate. The proposed method is characterized by 
robustness despite the degradation or modification on the watermarked content. Furthermore, the 
method can be extended to video, audio or other multimedia especially for multimedia databases 
in which the stored multimedia arc categorized by their contents or classes. 
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METHOD AND APPARATUS FOR EMBEDDING DIGITAL INFORMATION 
IN DIGITAL MULTIMEDIA DATA 

Field of Invention 

5 The present invention relates to a method and apparatus for embedding digital 

information in digital multimedia data. The invention also relates to a computer program 
product including a computer readable medium having recorded thereon a computer 
program for embedding digital information in digital multimedia data. 

10 Background of Invention 

Over recent decades there has been a rapid growth in computers and digital 
communications networks. With this growth, the illegal distribution, duplication and the 
manipulation of digital or digitised multimedia content have become more and more 
common place. Even though such activity may violate copyright laws, it is still 

15 widespread. There is thus a need to protect the rights of owners who own these original 
multimedia data. 

Similarly, there has also been a proliferation of exchange of sensitive data over 
public communications networks. Recently, digital signatures have been used to verify 
and authenticate the integrity of such data and identity the sender of the data. However 

20 there exists a need for invisible digital signature for images, which adequately represents 
the content of the images and further can be used for verifying whether the signed 
images have been modified. 

In addition, there has been an associated growth in multimedia databases. Usually 
the stored multimedia data needs to be annotated (captioning). If the annotation can be 

25 associated with the content permanently, it will be convenient and safe for some 
applications such as medical image database. 

Digital watermarking techniques can be applied to the above-mentioned 
applications, namely, for copyright protection, multimedia content authentication and 
multimedia content annotation. Although visible watermarks have been previously used, 

30 there is much promise for invisible digital watermarking techniques due to its 

unobtrusive nature. Regardless of particular applications, the following are the basic and 
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common requirements for invisible watermarks. The watermark should be unobtrusive 
and not annoying to bona-fide users. The watermark scheme should be robust to some 
reasonable distortions during storage and transmission (such as compression, noise 
addition, format conversion, and bit errors). The watermark scheme should also be 

5 robust to some artifacts caused by signal processing operations (such as noise reduction, 
filtering, sharpening, and colour & intensity variations) It should also be secure against 
typical attacks such as colluding pirates who combine multiple versions of the same 
multimedia content that are stamped with different watermarks. Erasing the watermark 
or replacing the watermark by another watermark should be technically difficult. 

10 Although these requirements are essential to practical applications, so far no one 

method or system is capable of fully satisfying the above conditions due to its so many 
unsolved problems and challenges. 

Aspects of Invention 

15 It is an object of the present invention to ameliorate one or more disadvantages of 

the prior art. 

According to one aspect of the invention, there is provided a method of embedding 
digital information in digital multimedia data, wherein said method includes the following 
steps; generating one or more transforms from the digital multimedia data or a portion of 

20 the digital multimedia data, wherein said transforms are dependent on the digital 

multimedia data or the portion of the digital multimedia data; generating transformed 
digital multimedia data by applying the one or more transforms to the digital multimedia 
data or the portion of the digital multimedia data; embedding the digital information in 
the transformed digital multimedia data; and generating the digital multimedia data 

25 embedded with the digital information by applying one or more inverse transforms of the 
one or more transforms to the embedded transformed digital multimedia data. 

According to still another aspect of the invention, there is provided a method of 
embedding digital information in digital multimedia data, wherein said method includes 
the following steps: generating one or more transforms from the digital multimedia data 

30 or a portion of the digital multimedia data, wherein said transforms are dependent on the 
digital multimedia data or the portion of the digital multimedia data; generating 
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transformed digital multimedia data by applying the one or more transforms to the digital 
multimedia data or the portion of the digital multimedia data; embedding the digital 
information in the one or more transforms or their inverses; and generating the digital 
multimedia data embedded with the digital information by applying one or more 
5 embedded inverse transforms to the transformed digital multimedia data. 

According to still another aspect of the invention, there is provided a method of 
embedding digital information in a digital image, wherein said method includes the 
following steps: partitioning the image into blocks of pixels; clustering the partitioned 
blocks into classes according to a homogeneity criteria; selecting one or more of the 

to clusters in accordance with HVS analysis; generating an associated Karhunen-Loeve (K- 
L) transform for each selected cluster, wherein the image blocks of each selected cluster 
are used as a vector population for the purposes of generating the associated K-L 
transform; applying the associated K-L transform to the image blocks of the selected 
cluster to produce corresponding transformed image blocks; embedding, for each 

is selected cluster, the digital information in the associated K-L transform; and applying, 
for each selected cluster, an inverse transform of the embedded associated K-L transform 
to the transformed image blocks of the selected cluster. 

According to still another aspect of the invention, there is provided a method of 
embedding digital information in a digital image, wherein said method includes the 

20 following steps: partitioning the image into blocks of pixels; clustering the partitioned 
blocks into classes according to a homogeneity criteria; selecting one or more of the 
clusters in accordance with HVS analysis; generating an associated Karhunen-Loeve (K- 
L) transform for each selected cluster, wherein the image blocks of each selected cluster 
are used as a vector population for the purposes of generating the associated K-L 

25 transform; applying the associated K-L transform to the image blocks of the selected 
cluster to produce corresponding transformed image blocks; embedding, for each 
selected cluster, the digital information in the selected transformed image blocks; 
applying, for each selected cluster, an inverse transform of the associated K-L transform 
to the embedded transform image blocks of the selected cluster. 

30 According to still another aspect of the invention, there is provided a method of 

embedding digital information in a digital image, wherein said method includes the 
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following steps: partitioning the image into blocks of pixels; clustering the partitioned 
blocks into classes according to a homogeneity criteria; selecting one or more of the 
clusters in accordance with HVS analysis; generating an associated Karhunen-Loeve (K- 
L) transform for each selected cluster, wherein the image blocks of each selected cluster 

5 are used as a vector population for the purposes of generating the associated K-L 
transform; applying the associated K-L transform to the image blocks of the selected 
cluster to produce corresponding transformed image blocks; selecting, for each selected 
cluster, either the associated K-L transform or its corresponding transformed image 
blocks for embedding; embedding, for each selected cluster, the digital information in 

10 the selected K-L transform or the selected transformed image blocks; applying, for those 
selected clusters in which the digital information has been embedded in the associated K- 
L transform, an inverse transform of the embedded associated K-L transform to the 
transformed image blocks of the selected cluster; and applying, for those selected 
clusters in which the digital information has been embedded in the transformed image 

15 blocks, an inverse transform of the associated K-L transform to the embedded transform 
image blocks of the selected cluster. 

According to still another aspect of the invention, there is provided a method of 
embedding digital information in a plurality of images, wherein the images are stored and 
classified into one or more classes and said method includes the following steps: 

20 generating one or more transforms from the images of said respective one or more 
classes, wherein said transforms are dependent on the images; generating transformed 
images by applying the one or more transforms to the images of said respective one or 
more classes; embedding the digital information in the transformed images; and 
generating the images data embedded with the digital information by applying one or 

25 more inverse transforms of the one or more transforms to the embedded transformed 
images. 

According to still another aspect of the invention, there is provided a method of 
embedding digital information in a plurality of images, wherein the images are stored and 
classified into one or more classes and said method includes the following steps: 
30 generating one or more transforms from the images of said respective one or more 
classes, wherein said transforms are dependent on the images; generating transformed 
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images by applying the one or more transforms to the images of said respective one or 
more classes; embedding the digital information in the one or more transforms or their 
inverses; and generating the images embedded with the digital information by applying 
one or more embedded inverse transforms to the transformed images. 
5 According to still another aspect of the invention, there is provided a method of - 

embedding digital information into video images including a plurality of frames, wherein 
said method includes the following steps: extracting one or more key frames from the 
video images; generating one or more transforms from the key frames or a part thereof, 
wherein said transforms are dependent on the key frames or the part thereof; generating 

io transformed key frames or part thereof by applying the one or more transforms to the 
key frames or the part thereof; embedding the digital information in the transformed key 
frames or part thereof; and generating the key frames or part thereof embedded with the 
digital information by applying one or more inverse transforms of the one or more 
transforms to the embedded key frames or part thereof 

15 According to still another aspect of the invention, there is provided a method of 

embedding digital information into video images including a plurality of frames, wherein 
said method includes the following steps: extracting one or more key frames from the 
video images; generating one or more transforms from the key frames or a part thereof, 
wherein said transforms are dependent on the key frames or the part thereof; generating 

20 transformed key frames or part thereof by applying the one or more transforms to the 
key frames or the part thereof; embedding the digital information in the one or more 
transforms or their inverses; and generating the key frames or part thereof embedded 
with the digital information by applying one or more embedded inverse transforms to the 
transformed key frames or part thereof 

25 According to still another aspect of the invention, there is provided a method of 

registering an image, wherein the method includes the following steps: extracting key 
points from the image and their invariant attributes; extracting other global information 
from the image; aligning the extracted key points and their invariant attributes with the 
global information; and encrypting the aligned information to provide an image 

30 signature. 
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According to still another aspect of the invention, there is provided a method of 
modifying a first image in accordance with an image signature, wherein the image 
signature includes encrypted key points of a second image and their invariant attributes 
and other aligned global information of the second image and the method includes the 
5 steps of: decrypting the image signature to obtain the key points and their invariant 
attributes and other global information of the second image; extracting key points and 
their invariant attributes from the first image; extracting other global information from 
the first image; matching the key points and their invariant attributes and other global 
information of the first and second images; and normalising the first image based on said 
10 matched key points and their invariant attributes and other global information. 

According to still another aspect of the invention, there is provided a method of 
extracting digital information from first digital multimedia data having embedded therein 
said digital information, wherein said method includes the following steps: deriving one 
or more embedded transforms from a reference file, wherein said embedded transforms 
1 5 are dependent on second digital multimedia data or a portion thereof; deriving 

transformed digital multimedia data from the reference file, wherein said transformed 
digital multimedia data is dependent on the second digital multimedia data or a portion 
thereof, generating transformed digital multimedia data by applying one or more of the 
derived embedded transforms to the first digital multimedia data or a portion thereof; 
20 and correlating the generated transformed digital multimedia with the derived 

transformed digital multimedia to check whether the digital information embedded in the 
first multimedia data is the same as that derived from the reference file. 

According to still another aspect of the invention, there is provided a method of 
extracting digital information from first digital multimedia data having embedded therein 
25 said digital information, wherein said method includes the following steps: deriving one 
or more transforms from a reference file, wherein said transforms are dependent on 
second digital multimedia data or a portion thereof; deriving transformed digital 
multimedia data from the reference file, wherein said transformed digital multimedia data 
is dependent on the second digital multimedia data or a portion thereof; generating 
30 embedded transformed digital multimedia data by applying one or more of the derived 
embedded transforms to the first digital multimedia data or a portion thereof; and 
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correlating the generated embedded transformed digital multimedia with the derived 
transformed digital multimedia so as to extract the digital information. 

According to still another aspect of the invention, there is provided a method of 
embedding digital information in a digital image, wherein said method includes the 
& following steps: partitioning the image into blocks of pixels; clustering the partitioned 
blocks into classes according to a homogeneity criteria; selecting one or more of the 
clusters in accordance with HVS analysis; averaging, for each selected cluster, the blocks 
of pixels; embedding, for each selected cluster, the digital information in the averaged 
blocks; and replacing, for each selected cluster, the blocks with the embedded averaged 

i o block for that cluster. 

According to still another aspect of the invention, there is provided a method of 
extracting digital information from an image having embedded therein said digital 
information, wherein said method includes; partitioning the image into blocks of pixels; 
clustering the partitioned blocks into classes according to a homogeneity criteria; 

15 selecting one or more of the clusters in accordance with information obtained from a 
reference file; retrieving, for each selected cluster, an average block of pixels stored in 
the reference file; averaging, for each selected cluster, the blocks of pixels of the image; 
and correlating, for each selected pixel, the averaged blocks of the image with the 
retrieved blocks to determine whether a watermark exists. 

20 According to still another aspect of the invention, there is provided an apparatus 

for implementing any one of the aforementioned methods. 

According to still another aspect of the invention there is provided a computer 
program product including a computer readable medium having recorded thereon a 
computer program for implementing any one of the methods described above. 

25 

Brief Description of the Drawings 

Embodiments of the invention are described with reference to the drawings, in 
which: 



30 FIG. 1 is a flow diagram of a method of embedding a watermark in a digital image in 
accordance with a preferred embodiment; 
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FIG. 2 shows a copyright protection system for protecting an owner's watermarked 
image in accordance with a preferred embodiment; 

FIG. 3 is a flow diagram of a method of generating a reference file that can be used for 
both image registration and for image authentication in accordance with a preferred 
5 embodiment; 

FIG. 4 is a flow diagram of a method of extracting a watermark from an image in 
accordance with a preferred embodiment. 

FIG. 5 shows a flow diagram of a method of registering and recovering an image for use 
in the extraction method shown in Fig. 4; 
1 o Fig. 6 is a flow diagram of a method of embedding a watermark in a digital image 
according to a second preferred embodiment; 

Fig. 7, there is shown a watermark extracting method for extracting a watermark 
embedded in accordance with the method of Fig. 6; and 

FIG. 8 is a block diagram of a general-purpose computer for implementing the preferred 
15 methods of FIGS. 1 to 7. 

Detailed Description 

The principles of the preferred method described herein have general applicability 
to a method of embedding a digital message within digital multimedia content. However, 

20 for ease of explanation, the steps of the preferred method are described with reference to 
a method of embedding a watermark within a still digital image. Though, it is not 
intended that the present invention be limited to the latter method. For example, the 
invention may have application to embedding an identification message within other 
multimedia content such as video, audio, print or film. The embedded identification 

25 message can also be used for copyright protection, content authentication and content 
annotation without departing from the scope and spirit of the invention. 

First Preferred Embodiment s) 

Fig. 1 is a flow diagram of a method of embedding a watermark in a digital image 
30 according to a preferred embodiment. The method commences at step 100 and any 
necessary initialisation processes are undertaken. In the next step 102, an original digital 
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image is input by a user for processing. The original digital image is a pixel-map 
representation which includes a numerical representation of the particular colour for each 
pixel location in a rectangular array. Any numerical representation of colour can be used 
and can be expressed as a sequence of one or more numbers. Also, other information 
5 may be input at this stage, such as header information. Such header information can 
include the size of the image, resolution etc. 

The input original image is then preprocessed in step 104. The preprocessing may 
take the form of filtering, noise estimation and so on. After the preprocessing step 104, 
the method continues with a block and clustering step 106. 

io In the block and clustering step 106, the selected original image is divided into N 

non-overlapping blocks of size nxn. Preferably, these blocks are either 4x4 pixels or 8x8 
pixels in size. In the event the original image is not a multiple of the image block size, a 
portion of the original image which is a multiple can be used by the method and the 
remaining portion disregarded. The block and cluster step 106 allocates these image 

15 blocks to clusters (sometimes known as classes) according to the details of the image 
block content and a homogeneity criteria. Thus the image blocks within each cluster will 
have similar amount of and type of content. Preferably, the clusters are classified 
according to their complexity or texture content. This classification is achieved in the 
following manner. Firstly, feature measures are extracted from each image block which 

20 are representative of the complexity of the content. Some preferred feature measures are 
grey scale, colour variation or texture measures. Afterwards, the clustering process 
classifies the image blocks according to their feature measures. The clustering process 
preferably utilises the k-means clustering method. Alternatively, the preferred method 
can use region growing, or split-merge clustering processes. In this way, the block and 

25 clustering step allocates the image blocks to a plurality of clusters. After the block and 
clustering step 106 is completed, the processing continues at step 108. 

In step 108, a cluster selection process is undertaken. In this selection process, an 
analysis based on the Human Vision System (HVS) is applied to all clusters to determine 
whether and by how much the watermark can be embedded. Preferably, those clusters 

30 which satisfy the HVS requirements are selected for watermark embedding. The 

perceptual redundancies inherent in a still image are basically due to inconsistency in the 
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sensitivity of the HVS to stimuli of varying levels of contrast and luminance changes in 
the spatial domain. Thus, by utilising the HVS model in the preferred method it is 
possible to embed more information while not overall affecting image quality. Or in other 
words, the embedded watermark information is invisible to human eyes. In the preferred 
s method, the HVS model is used to select the clusters and the embedding strength. For 
example, the embedding strength is determined as a proportion of the contrast of the 
image content of the cluster. Preferably, the HVS model used is that described in the 
paper "A perceptually tuned subband image coder based on the measure of Just- 
Noticeable-Distortion profile" by C. H. Chou and Y. C. Li, IEEE Trans. On circuits and 
, o Systems for Video Technology, Vol. 5, No. 6, pp467-476, 1 995 (herein incorporated by 
reference). In another embodiment, if there are a number of clusters which satisfy these 
HVS requirements, then one or more clusters can be selected by the user for embedding. 
Alternatively, different clusters can be embedded with different watermarks with the 
same seeds to increase the security and robustness of the watermarking. After the 
1 5 clusters have selected the preferred method proceeds to the next step 110. 

In the next step 110, one or more Karhunen-Loeve transforms (KLT) are applied 
to the image blocks of one or more selected clusters. These KLT transforms are each 
derived from and applied to a corresponding selected cluster. It is noted that, up until 
today, the statistically optimum, signal-dependent transform, KLT is still believed to be 
20 impractical for image processing. This is due to the time-consuming computations 

needed for determining the data covariance matrix (related to eigen-vectors). However 
in the preferred method, this will not be a crucial problem any more. Firstly, the KLT 
adopted in the preferred method is the discrete KLT (also called the Principle 
Components Analysis) where there have been some fast implementations developed 
25 recently. Secondly, unlike image compression or image coding, for watermarking there is 
no need to transform all blocks or segmented regions in the images. The fast clustering 
processing associated with the cluster selection step 106 allows the reduction of the 
computational burden to an acceptable and practical level. Namely, the KLT transforms 
need not be applied over all the image, but only to the selected clusters, 
so In the step 1 1 0, a KLT transform is separately derived and applied for each cluster. 

Firstly, each non-overlapping block in the / th cluster (segment), / = 1,A , N , is mapped 



WO 00/39954 PCT/SG98/001 1 0 

- 11 - 

to a d -dimensional vector X, y , where d - 3n 2 , by sequentially scanning each location 
in the block and taking its corresponding tricomponent colour signals. (Note: for 
monochrome image, d = n 2 ). Here N denotes the total number of clusters to be 
transformed. Thus each block in the same cluster can be implicitly modelled as separate 
5 multivariate Gaussian Markov random fields. The sets of vectors X iy created in this way 
are assumed to have jointly zero-mean Gaussion distributions. The dxd covariance 
matrix R ( of each such stationary zero-mean Gaussion source, /' = 1,A , N , is estimated 
as 



10 



R, = — — 5^X«Xg (1) 



Where M i corresponds to the number of blocks in the / th cluster. The preferred method 
then calculates Eigenvalues of the covariance matrix R, in accordance with 

15 R,c tt =V tt (2) 

Where e ik is the k th eigenvector (k = 1,K ,/? ) of the / th cluster and ?. ik is its 
corresponding eigenvalue. The eigenvectors calculated from (2) are orthonormal, ie., 

20 ef,e l7 = 0 (3) 

e*e*=l (4) 

Subsequently, the eigenvectors are arranged such that X n > X n > A > X id and the first 

^ eigenvectors, where < d , can be used to form the transform matrix 

25 A, = [e n e l2 A J" , where d is the dimension of vector X i} . . Preferably however, 

all the eigenvectors are used to form the transform matrix, viz L, = d The reason for the 
latter is that the preferred method is primarily concerned with watermarking and not 
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compression. Each vector X tf from the / th image cluster can then be processed by the 
linear transform: 

Y,=A,.X, ; = 1,K,M, (5) 

After step 110, the preferred method proceeds to the next step 1 12. In step 1 12, a 
user decides whether the watermark is to be embedded in either the eigenvector domain 
A, or the coefficients of vectors Y f> . Preferably, the user selects either the eigenvector 
domain A, or the coefficients of vectors for the entire image, viz for all clusters. 
However in another embodiment, the user is able to select the eigenvector domain A,, 
for one cluster and the coefficients of vectors Y^. for another cluster. This will result in 
benefits in the security of the watermarking method. In a still further embodiment, the 
method automatically and exclusively embeds the watermark in the eigenvector 
domain A, . Namely, the selection is automatic and no input from a user is required. 
Similarly, the method can automatically and exclusively embed the watermark in the 
coefficients of vectors Y (> .. In still further embodiment, the actual selection process is 
dependent on the requirements of the particular application. After step 112, the method 
continues processing at step 114. 

In step 1 14, the method embeds the watermark in the selected domain, namely 
either the eigenvector domain A, or the coefficients of vectors Y ff In the event the 
preferred method embeds the noise-like watermark into the coefficient domain, the 
following embedding process is performed. A property of the coefficients of the 
vectors Y 0 - are that they are completely uncorrected, as can be seen from eqn (6). 



0 A 



25 



0 A,., 



0 
0 



M M M M 
0 0 A ^ 



(6) 
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where Z n > A i2 > A > X id and are the eigenvalues of the co variance matrix R, 

The preferred method embeds the watermark in the first L,- coefficients of each vector 

Y iy where 

5 L t = argmax^ : X ik < D t (7) 

k 

The parameter in (7) represents the allowed distortion within the cluster. Its value 
will be defined for each cluster separately and decided by the results of HVS analysis. 
Thus during this embedding process the vectors Y. are changed to Y~ . 

In the event the preferred method embeds the noise-like watermark into the 
io eigenvector domain, the watermark will be embedded into the first Z, eigenvectors of A, 
in accordance with eqn (7). Thus during this embedding process the transform A, is 
changed to A ; . As the eigenvectors of the transform matrix are orthogonal to each 

other, the generated watermark W t should be orthogonalized with respect to the 
eigenvectors before embedding. 
15 In another embodiment, the transform A, may be first inverted and then embedded 

with the watermark. 

All the parameters of the selected domain are embedded, by modulating the 
parameters by a noise-like watermark. This modulation is achieved in the following way 

20 P = f(F,PN(W)) (8) 

Where / is a function of watermarking modulation, P and P are parameters after and 
before watermarking, and PN(W) is generated noise-like watermark. 

The content of image itself, the owner's private key of an encryption algorithm, 
25 embedding time or other information related to the owner or the buyer can be used to 
form the seeds of the embedded watermarks. Furthermore error correction encoding can 
be used to increase the robustness of the preferred method. The preferred method can 
use any one of a number of typical methods to embed a watermark (modulation). A 
simple example of such a embedding method is as follows 
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/>=;>+<*, (9) 

After the completion of the watermark embedding step 1 14, the process continues 
at step 1 16. In step 116, the respective inverse KLT transforms are separately applied to 
their respective clusters. Where the watermark has been embedded in the eigenvector 
domain, the "embedded" inverse KLT transform is applied to the vectors. Thus the 
watermarked image blocks are transformed by 

X,=A;% j = lK,M, (10) 

Where the watermark has been embedded in the coefficients of the vectors, the 
inverse KLT is applied to the embedded vectors. Thus the watermark image blocks are 
transformed by 

j = \TLM, (11) 



These resultant image blocks form the basis of the watermarked image. To retrieve 
the watermark from the watermark image or to authenticate the watermark image a 
20 reference file is needed. This reference file may be the original image itself or data based 
on the original image. Such data can include the size of the image, the resolution, the 
clusters selected and their associated KLT transform, modulation technique and owner 
information. Consequently, after the original image is preprocessed in step 104, the 
preferred method stores 122 the original image or the derived data as the reference file. 
25 In step 122, the reference file is normally held in a secure location and maybe encrypted. 

After completion of inverse transform KLT 116, the preferred method continues at 
step 1 1 8, where the watermarked image is output. The method terminates at step 118. 

The use of the KLT transform as the watermark embedding domain has the 
following advantages. Firstly, the KLT transform is a kind of signal-dependant 
30 transform. That means the watermark to be embedded will follow any changes of image 
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content. Even if the watermark is the same for all different images, the real form of the 
embedded watermark is different. Thus in this sense, the embedded digital watermark is 
dependent upon the image content and can be used to authenticate an image. 
Furthermore, whilst the type of transform used to generate the image may be known, the 
5 original watermark cannot be defeated by inverse transforming the watermarked image 
data to derive the original data, since this involves the use of a different inverse 
transform. That is, because the watermarked data is different from the original data. 
Likewise, attempts to fraudulently claim ownership on image data already watermarked 
by the preferred method will be defeated because a different optimal transform 

10 dependent upon the watermarked image data will be involved. Still further, the KLT 
transform presents an extra choice to select the embedding locations either in the eigen- 
vector domain (basis function of KLT transform) or in the coefficient domain of KLT 
transform. This means that watermarking based on KLT can be done in a much secure 
and flexible way than other methods. In addition, the KLT transform can provide a more 

15 effective energy-compact ability more than any other transform such as DCT, DST, FFT, 
etc. As there is significant redundant information contained in an image, applying a KLT 
transform to an image means that there is more space for embedding the watermark. 
Also, unlike most of common watermarking schemes which only embed the watermark 
into the luminance channel, the KLT transform can directly & jointly process colour 

20 images. It presents a flexible way to embed a watermark directly into all colour channels. 
In addition, the embedded messages will persist through image modifications such as 
scaling, rotating, printing-rescanning, colour & intensity variation. 

Turning now to Fig. 2 there is shown a copyright protection system for protecting 
an owner's watermarked image in accordance with a preferred embodiment. Firstly, the 

25 owner of the image 200 sends his request 201 to a Copyright Management Centre 

(CMC) 202 for registration. The CMC 202 provides the owner of the image 200 with a 
watermark embedding tool 203 to embed his private watermark in his images 205. 
During this procedure, a reference file 206 is generated and sent to CMC 202. The 
reference file 206 is based on a content analysis 207 of the owner's image 205. After 

30 watermarking 204, the CMC 202 will activate an Internet Spider 208 to protect the 
owners watermarked images 210. Of course, there are so many variant applications 
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related to copyright protection, content authentication and content annotation. 
Preferably, the watermarking engine is that described with reference to FIG. 1, namely 
the watermarking engine is based on KLT transforms. This watermarking engine can be 
constructed as a kernel and integrated into various copyright protection systems either 

5 for public use or for private use. 

Turning now to Fig. 3, there is shown a flow diagram of a method of generating a 
reference file that can be used for both image registration and for image authentication. 
This method of generating a reference file may be used in the method of watermarking 
an image as shown in Fig. 1 . The method commences at step 300 and continues at steps 

10 302 and 304 where an image is input and preprocessed. In the next step 306, the method 
summarises and extracts some key points of the preprocessed image. These key points 
must be invariant to some typical modifications such as affine distortions, colour or 
intensity changes etc. Alternatively, some variations of the key points can be allowed by 
adopting a global affine matching algorithm in the registration procedure. Then the 

is invariant attributes are calculated 308. These attributes are associated with other relative 
information such as maximum and minimum colour values, clusters and the number of 
clusters, and KLT transform matrices etc. Finally this associated information is encrypted 
by some cryptography algorithms 3 10 to form the reference file 412. The method 
terminates at step 3 14. In this way, the watermarked image is registered with reference 

20 to the properties of the original image, which can be used in the method of extracting the 
watermark from the watermarked image. Alternatively, the reference file can be used for 
proving whether and by how much the watermarked image has been tampered with. 

Turning now to Fig. 4, there is shown of a watermark extracting method in 
accordance with a preferred embodiment. The method commences at step 400, where 

25 any necessary parameters are initialised. In the next step 402, a watermarked image is 
input by a user for watermark extraction. Afterwards in step 404, the user then inputs a 
reference file 403 of the original image and/or associated parameters of the type as 
previously described. The watermarked image is then normalised 406 based on the 
parameters retrieved from the reference file 403. This normalised image is partitioned 

30 into blocks 106 and then clustered 108, in a similar manner as the watermarking 

procedure as described with reference to Fig.l. In step 108, any parameters necessary 
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for the selection of the clusters are retrieved from the reference file 403. After step 108, 
the method continues with step 408 where the relevant domain is selected from data 
from the reference file. In the case where the domain selected is the coefficient domain, 
the next step 410 applies the transform A, to selected image blocks X iy of the test image 

5 to obtain the coefficients of vectors Y z> . The images blocks X iy of the test image are 

decided by the reference file or owner's private key. The transform A. is also obtained 

from the reference file. Finally, a correlation 412 is undertaken between the last 

mentioned coefficients of vectors Y f> and the coefficients of vectors Y iy obtained from 

the reference file and the watermark is extracted. The extracted watermark is then 
io compared with the watermark obtained from the reference file. In the case where the 
domain selected is the transform domain, the step 410 applies the embedded 

transform A,, to selected image blocks X,., of the test image to obtain the coefficients Y jy . 
A correlation 412 is then undertaken between the last mentioned coefficients of vectors 
Y iy and the coefficients of vectors Y (> obtained from the reference file so as to check 

15 whether there exists a watermark. If the coefficients are the same then the watermark is 
the same as was originally embedded. The extracted watermark is then output in step 
418 and the method then terminates. 

Turning now to Fig. 5 there is shown a flow diagram of a method of registering 
and recovering an image for use in the extraction method shown in Fig. 4. As a 

20 watermarked image may undergo some modifications during it's lifetime, an operation 
like image registration is used to estimate the modification parameters of the 
watermarked image. The method commences at step 500 and continues at step 502 
where a watermarked image is input. The watermarked image is then preprocessed 504 
and summarised 506 to obtain some key features or points. Once these key points are 

25 optimally selected, their invariant feature attributes are computed. Meanwhile relative 
parameters are obtained from the reference file 508 by an information recovering 
procedure such as decryption. Such a reference file may be obtained by a method as 
described with reference to Fig. 3. In the next step, a rule-based global optimal search 
510 is undertaken between the obtained invariant feature attributes of the watermarked 
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image and reference file. For example, if some feature points within a neighbour are 
missed, then a chopping operation may be applied onto the watermarked image. The 
matching is mainly based on their invariant moments of the detected feature points. 
Based on the estimated parameters, the watermarked image is the normalised 512 to 

5 recover an image similar to the original image. The normalisation method can be that 
which is described in J. Flusser, "An adaptive method for image registration", Pattern 
Recognition, Vol 25, No. 1, pp45-54, 1992. The method normalises the tested image by 
extracting the key content from the tested image, measuring the modified parameters of 
the tested image with reference to the reference file and then normalising the tested 

1 o image with respect to these modified parameters. 

The present invention is suitable for use in a number of different and widely 
varying applications, such as the copyright protection system as previously described. 
This system can provide an approach to track subsequent use of digital images and then 
present evidence for copyright protection, or alternatively provide a display warning 

15 users. 

The present invention also can be used for content authentication. In this 
application, after summarising the image, the extracted key description about the image 
can be encrypted by the owner's private key. The buyer can then verify the authenticity 
of the image by using a public key. Based on the rule-based image registration method, 

20 the results of the authentication include the information about not only whether the given 
image is altered or not, how much it is altered, but also which kind of modification the 
said image undergoes. The application may also inform the owner of the images what 
kinds of attacks others have used on his or hers images. A digital signature may also be 
embedded into the image for verifying the authenticity of the content provider. 

25 In addition, the present invention also can be used for content annotation. It is 

believed that the annotation done by watermarking will be helpful for multimedia 
database based on the current technologies. 

The present invention is also particularly suitable for watermarking the images that 
has been stored in the multimedia database no matter what kind of applications it is used 

30 for (copyright protection, authentication and annotation). As in this case, the image will 
belong to some classes or categories. For example, a face database in the police station, 
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thus a optimal basis matrix of KLT can be obtained based on the whole images from a 
class. The images are then embedded in a similar manner as described with reference to 
Fig. 1 . In this way, it is very difficult for the users or the attackers to compute out the 
exact basis matrix. Without the exact basis matrix, it is unlikely to decode or to remove 
5 the embedded watermark. 

Furthermore, the present invention can be used in relation to other media such as 
video and audio. For instance, in a video file, there is a strong relationship between 
consecutive frames. A technique called key frame extraction has been shown to be an 
effective approach to video indexing or summarising. By applying the KLT to hide the 
io information directly into video file, the optimal basis matrix of the KLT can be computed 
out not only based the clustered blocks within each frame but also based on frames 
represented by the same key frame. Based on KLT, the data hiding can be implemented 
in a more secure and flexible way. 

is Second Preferred Embodiment^ 

Fig. 6 is a flow diagram of a method of embedding a watermark in a digital image 
according to a preferred embodiment. The method commences at step 100 and any 
necessary initialisation processes are undertaken. In the next step 102, an original digital 
image is input by a user for processing. The original digital image is a pixel-map 

20 representation which includes a numerical representation of the particular colour for each 
pixel location in a rectangular array. Any numerical representation of colour can be used 
and can be expressed as a sequence of one or more numbers. Also, other information 
may be input at this stage, such as header information. Such header information can 
include the size of the image, resolution etc. Steps 102 to 108 of this method are the 

25 same as those described with reference to steps 102 to 108 of Fig. 1 and will not be 
described further. 

In step 602 of the method, the image blocks of each cluster / are averaged. These 
averaged blocks are each then embedded 604 with a watermark. An average (mean) 
block for Mj image blocks By ( j = 1,2,....M) that belong to cluster / is determined by 
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and a difference block is obtained by 

X y -=B*- B, (13) 

The average block can be used to embed the watermark if it satisfies the HVS 
condition. If there are more than one suitable clusters then the user can select the cluster 
5 used for embedding. The averaging and embedding process is described in more detail 
below with reference to the following alternative embodiments. 

In the first embodiment, the embedding is achieved by an orthogonalization 

process. Firstly, the average mean block B, is aligned as a (n x n)xl vector. This 

averaged mean block B, is saved to a reference file for subsequent watermark 

10 extraction. The initial watermark W t - then is orthogonalized with respect to B* to obtain 
the final watermark W f . in accordance with the following: 

W f (14) 
where B. is the unit vector along B, . 

Next, a smaller scaled version of Wis added back to B, to obtain a watermarked 
is average block B, in accordance with the following: 

B^Bj+cWj, where c is a small constant to control the strength of the 

watermark. (15) 
Finally in step 606, all the image blocks of the cluster are replaced with the 

watermarked average block B, to reconstruct a new image (watermarked image). The 
20 cluster criteria is chosen (108) so that the distortion caused by replacing all the image 
blocks within the cluster with the watermarked average block would be invisible to the 
human eye. 

In the second embodiment, the embedding is achieved by a spread spectrum 

process. In this embodiment, the average mean block B, is first aligned as a {n x n) x I 
25 vector. The initial watermark W t is shaped to the same rtxn length and with gaussian 



WO 00/39954 



- 21 - 



PCT/SG98/O01I0 



distribution (zero mean value). A small scaled version of Wi is added back to B, to 
obtain a watermarked average block B, in accordance with the following: 

B, =B, •(} +cW f ) 9 where c is a small constant to control the strength of the 
watermark. (16) 

5 

Finally in step 606, all the image blocks of the cluster are replaced with the 
watermarked average block B, to reconstruct a new image (watermarked image). The 
cluster criteria is chosen (108) so that the distortion caused by replacing all the image 
blocks within the cluster with the watermarked average block would be invisible to the 
10 human eye. 

After the completion of step 606, the watermarked image is outpoured and the 
process terminates at step 610. 

Turning now to Fig. 7, there is shown a watermark extracting method for 
extracting a watermark embedded in accordance with the method of Fig. 6. The method 
15 commences at step 700, where any necessary parameters are initialised. The test image is 
then input 102 3 preprocessed 104 and block and clustered 106 in a similar manner as that 
of the embedding method of Fig. 6. After completion of step 106, one or more clusters 
are selected for processing. The clusters selected are determined from information stored 
in the reference file 714. These clusters correspond to those selected in the embedding 

20 method. In the next step 706, the average mean block B, is obtained from the reference 
file. In this step, the watermark W, is also generated by the owner's key. The average 

mean block B, (from the reference file) is then aligned as a (w x n) x J vector. The 

watermark W f is then orthogonalised with respect to B, to obtain the original watermark 
in accordance with the following: 

25 W Q = W i - QV f • B, )B, where B, is the unit vector along B, . (17) 
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Similarly in the next step 708, a watermarked mean block J,, is then obtained from the 
test image. In the event, the watermark has been embedded by the orthogonal process, 

then watermarked mean block J,, is orthogonalised with respect to Bi to obtain another 
vector Xi. Finally in step 710, a correlation is undertaken between W, and X, to check 

5 whether the watermark exists. In the event, the watermark has been embedded by the 
spread spectrum process, then the DC component of J, is removed. In this case a 
correlation is undertaken between Wj and J, to check whether the watermark exists. 
After the correlation is completed the method terminates at step 712. 

In a still further embodiment, the method of embedding and extraction of the 

10 second preferred embodiment may include a method of registering and recovering an 
image as shown in Figs. 3 and 5. In this way, modifications that may subsequently occur 
to the embedded image are compensated by normalisation. In a still further embodiment, 
the first and second preferred embodiments may form part of the same watermarking 
engine. In the latter embodiment, a user's key is used to select the watermarking method 

15 to be used on a particular image. The user's key is an encrypted random key, which is 
divided into three parts. The first part is used to select the watermarking embedding 
space ( mean block, the transform or vector coefficients), the second is used to select the 
watermark extraction method (eg. de-orthogonalization or correlation) and the third part 
is used as the seed to generate the initial watermark Wj. In addition, the user's key can be 

20 used to select the cluster from the one or more clusters that meet the HVS criteria. 

Preferred Embodiment of Apparatusfs) 

The methods of Figs 1,3, 4, 5, 6 and 7 is preferably practiced using a conventional 
general-purpose computer, such as the one shown in Fig. 8 wherein the processes of 

25 Figs. 1,3, 4, 5 , 6 and 7 may be implemented as software executing on the computer. In 
particular, the steps of methods of embedding and extracting a watermark are effected by 
instructions in the software that are carried out by the computer. The software may be 
divided into two separate parts; one part for carrying out the extraction and embedding 
methods; and another part to manage the user interface between the latter and the user. 

30 The software may be stored in a computer readable medium, including the storage 
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devices described below, for example. The software is loaded into the computer from 
the computer readable medium, and then executed by the computer. A computer 
readable medium having such software or computer program recorded on it is a 
computer program product. The use of the computer program product in the computer 
5 preferably effects an advantageous apparatus for extracting and embedding a watermark 
in accordance with the embodiments of the invention. 

The computer system 800 consists of the computer 802, a video display 816, and 
input devices 818, 820. In addition, the computer system 800 can have any of a number 
of other output devices including line printers, laser printers, plotters, and other 

10 reproduction devices connected to the computer 802. The computer system 800 can be 
connected to one or more other computers via a communication interface 808b using an 
appropriate communication channel 830 such as a modem communications path, a 
computer network, or the like. The computer network may include a local area network 
(LAN), a wide area network (WAN), an Intranet, and/or the Internet 

15 The computer 802 itself consists of a central processing unit(s) (simply referred to 

as a processor hereinafter) 804, a memory 806 which may include random access 
memory (RAM) and read-only memory (ROM), input/output (IO) interfaces 808a, 808b 
& 808c, a video interface 810, and one or more storage devices generally represented by 
a block 812 in Fig. 8. The storage device(s) 812 can consist of one or more of the 

20 following; a floppy disc, a hard disc drive, a magneto-optical disc drive, CD-ROM, 
magnetic tape or any other of a number of non-volatile storage devices well known to 
those skilled in the art. Each of the components 804 to 812 is typically connected to one 
or more of the other devices via a bus 814 that in turn can consist of data, address, and 
control buses. 

25 The video interface 8 1 0 is connected to the video display 8 1 6 and provides video 

signals from the computer 802 for display on the video display 816. User input to 
operate the computer 802 can be provided by one or more input devices 808b. For 
example, an operator can use the keyboard 818 and/or a pointing device such as the 
mouse 820 to provide input to the computer 802. 

30 The system 800 is simply provided for illustrative purposes and other 

configurations can be employed without departing from the scope and spirit of the 
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invention. Exemplar)' computers on which the embodiment can be practiced include 
IBM-PC/ ATs or compatibles, one of the Macintosh (TM) family of PCs, Sun 
Sparcstation (TM), or the like. The foregoing are merely exemplary of the types of 
computers with which the embodiments of the invention may be practiced. Typically, 

5 the processes of the embodiments, described hereinafter, are resident as software or a 
program recorded on a hard disk drive (generally depicted as block 812 in Fig. 8) as the 
computer readable medium, and read and controlled using the processor 804. 
Intermediate storage of the program and pixel data and any data fetched from the 
network may be accomplished using the semiconductor memory 806, possibly in concert 

1 o with the hard disk drive 812. 

In some instances, the program may be supplied to the user encoded on a 
CD-ROM or a floppy disk (both generally depicted by block 812), or alternatively could 
be read by the user from the network via a modem device connected to the computer, 
for example. Still further, the software can also be loaded into the computer system 800 

15 from other computer readable medium including magnetic tape, a ROM or integrated 
circuit, a magneto-optical disk, a radio or infra-red transmission channel between the 
computer and another device, a computer readable card such as a PCMCIA card, and 
the Internet and Intranets including email transmissions and information recorded on 
websites and the like. The foregoing are merely exemplary of relevant computer 

20 readable mediums. Other computer readable mediums may be practiced without 
departing from the scope and spirit of the invention. 

The foregoing only describes a small number of embodiments of the present 
invention, however, modifications and/or changes can be made thereto by a person 
skilled in the art without departing from the scope and spirit of the invention. For 

25 example, signal dependent transforms other than KLT transforms may used in the 
preferred methods. The present embodiments are, therefore, to be considered in all 
respects to be illustrative and not restrictive. 
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1 . A method of embedding digital information in digital multimedia data, wherein said 
method includes the following steps: 

generating one or more transforms from the digital multimedia data or a portion of the 
5 digital multimedia data, wherein said transforms are dependent on the digital multimedia 
data or the portion of the digital multimedia data; 

generating transformed digital multimedia data by applying the one or more transforms 
to the digital multimedia data or the portion of the digital multimedia data; 
embedding the digital information in the transformed digital multimedia data; and 
10 generating the digital multimedia data embedded with the digital information by applying 
one or more inverse transforms of the one or more transforms to the embedded 
transformed digital multimedia data. 

2. A method as claimed in claim 1 , wherein said digital multimedia data is an image. 

3. A method as claimed in claim 2, wherein the method further comprises the 
following steps for selecting said portion of said digital multimedia data: 
partitioning the image into blocks; 

clustering the partitioned blocks into classes according to a homogeneity criteria; and 
selecting one or more of the clusters as said portion of said digital multimedia data in 
accordance with HVS analysis. 

4. A method as claimed in claim 3, wherein the selecting step is determined in 
accordance with HVS analysis and a user's selection. 

25 

5. A method as claimed in claim 1, wherein said transform is a Karhunen-Loeve 
transform. 

6. A method as claimed in claim 1, wherein said digital multimedia data is an image 
30 and said one or more transforms are Karhunen-Loeve transforms and said digital 

information is embedded in the first L, coefficients of the transformed digital 
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multimedia data, where the number L, is determined in accordance with HVS 



7. A method as claimed in claim 1 , wherein said digital information is a watermark. 

8. A method as claimed in claim 1, wherein said method further includes the 
following step: 

storing the digital multimedia data as a reference file together with associated 
information. 

9. A method as claimed in claim 2, wherein said method further includes the 
following steps 

extracting invariant representations of the image; 

storing said extracted representations together with associated information in a 
reference file. 

10. A method as claimed in claim 2, wherein said method further includes the 
following steps 

extracting key features from said image, and 

storing the key features together with associated information. 

11. A method as claimed in claim 10, wherein said key features include: the positions 
of the key features, invariant attributes of the key features, maximum and minimum 
colour values of the image, the size of the image. 

12. A method as claimed in claim 1 , wherein said digital information is a copyright 
notice. 



analysis. 



13. A method as claimed in claim 1, wherein said digital information is an annotation 
30 of said digital multimedia data. 
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14. A method as claimed in claim 1, wherein said digital information is authentication 
information verifying said digital multimedia data. 

15. A method of embedding digital information in digital multimedia data, wherein said 
5 method includes the following steps: 

generating one or more transforms from the digital multimedia data or a portion of the 
digital multimedia data, wherein said transforms are dependent on the digital multimedia 
data or the portion of the digital multimedia data; 

generating transformed digital multimedia data by applying the one or more transforms 
io to the digital multimedia data or the portion of the digital multimedia data; 

embedding the digital information in the one or more transforms or their inverses; and 
generating the digital multimedia data embedded with the digital information by applying 
one or more embedded inverse transforms to the transformed digital multimedia data. 

is 16. A method as claimed in claim 1 5, wherein said digital multimedia data is an image. 

1 7. A method as claimed in claim 1 5, wherein the method further comprises the 
following steps for selecting said portion of said digital multimedia data: 
partitioning the image into blocks; 

20 clustering the partitioned blocks into classes according to a homogeneity criteria; and 
selecting one or more of the clusters as said portion of said digital multimedia data in 
accordance with HVS analysis. 

18. A method as claimed in claim 17, wherein the selecting step is determined in 
25 accordance with HVS analysis and a user's selection. 

19. A method as claimed in claim 15, wherein said transform is a Karhunen-Loeve 
transform. 

30 20. A method as claimed in claim 15, wherein said digital multimedia data is an 
image and said one or more transforms are Karhunen-Loeve transforms and said 
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digital information is embedded in the first L t eigenvectors of the Karhunen-Loeve 
transforms, where the number L, is determined in accordance with HVS analysis. 

21 . A method as claimed in claim 1 5, wherein said digital information is a watermark. 

22. A method as claimed in claim 1 5, wherein said method further includes the 
following step: 

storing the digital multimedia data as a reference file together with associated 
information. 

23. A method as claimed in claim 16, wherein said method further includes the 
following steps 

extracting invariant representations of the image; 

storing said extracted representations together with associated information in a 
reference file. 

24. A method as claimed in claim 16, wherein said method further includes the 
following steps 

extracting key features from said image; and 

storing the key features together with associated information. 

25. A method as claimed in claim 24, wherein said key features include: the positions 
of the key features, invariant attributes of the key features, colour values of the key 
features, maximum and minimum colour values of the image, the size of the image. 

26. A method as claimed in claim 15, wherein said digital information is a copyright 
notice. 

27. A method as claimed in claim 1 5, wherein said digital information is an annotation 
of said digital multimedia data. 
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28. A method as claimed in claim 14, wherein said digital information is authentication 
information verifying said digital multimedia data. 

29. A method of embedding digital information in a digital image, wherein said method 
includes the following steps: 

partitioning the image into blocks of pixels; 

clustering the partitioned blocks into classes according to a homogeneity criteria; 

selecting one or more of the clusters in accordance with HVS analysis; 

generating an associated Karhunen-Loeve (K-L) transform for each selected 
cluster, wherein the image blocks of each selected cluster are used as a vector population 
for the purposes of generating the associated K-L transform; 

applying the associated K-L transform to the image blocks of the selected cluster 
to produce corresponding transformed image blocks; 

embedding, for each selected cluster, the digital information in the associated K-L 
transform; and 

applying, for each selected cluster, an inverse transform of the embedded 
associated K-L transform to the transformed image blocks of the selected cluster. 

30. A method as claimed in claim 29 : wherein said digital information is embedded 
in the first L, eigenvectors of the Karhunen-Loeve transforms, where the number L, is 
determined in accordance with HVS analysis. 

31. A method as claimed in claim 29, wherein said digital information is a noise-like 
watermark. 

32. A method as claimed in claim 29, wherein the selecting step is determined in 
accordance with HVS analysis and a user's selection. 



30 



33. A method of embedding digital information in a digital image, wherein said method 
includes the following steps: 

partitioning the image into blocks of pixels; 
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clustering the partitioned blocks into classes according to a homogeneity 
criteria; 

selecting one or more of the clusters in accordance with HVS analysis; 
generating an associated Karhunen-Loeve (K-L) transform for each selected 



5 cluster, wherein the image blocks of each selected cluster are used as a vector population 
for the purposes of generating the associated K-L transform; 

applying the associated K-L transform to the image blocks of the selected cluster 
to produce corresponding transformed image blocks; 

embedding, for each selected cluster, the digital information in the selected 
1 o transformed image blocks; and 

applying, for each selected cluster, an inverse transform of the associated K-L 
transform to the embedded transform image blocks of the selected cluster. 

34. A method as claimed in claim 33, wherein said digital information is embedded in 
1 5 the first Li coefficients of the transformed image blocks, where the number L, is 

determined in accordance with HVS analysis. 

35. A method as claimed in claim 33, wherein said digital information is a noise-like 
watermark. 

20 

36. A method as claimed in claim 33, wherein the selecting step is determined in 
accordance with HVS analysis and a user's selection. 

37. A method of embedding digital information in a digital image, wherein said method 
25 includes the following steps: 



partitioning the image into blocks of pixels; 



clustering the partitioned blocks into classes according to a homogeneity 
criteria; 

selecting one or more of the clusters in accordance with HVS analysis; 



0 
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10 



15 



20 



generating an associated Karhunen-Loeve (K-L) transform for each selected 
cluster, wherein the image blocks of each selected cluster are used as a vector population 
for the purposes of generating the associated K-L transform; 

applying the associated K-L transform to the image blocks of the selected cluster 
to produce corresponding transformed image blocks; 

selecting, for each selected cluster, either the associated K-L transform or its 
corresponding transformed image blocks for embedding; 

embedding, for each selected cluster, the digital information in the selected K-L 
transform or the selected transformed image blocks; 

applying, for those selected clusters in which the digital information has been 
embedded in the associated K-L transform, an inverse transform of the embedded 
associated K-L transform to the transformed image blocks of the selected cluster; and 

applying, for those selected clusters in which the digital information has been 
embedded in the transformed image blocks, an inverse transform of the associated K-L 
transform to the embedded transform image blocks of the selected cluster. 

38. A method as claimed in claim 37, wherein said digital information is embedded 
in the first L t eigenvectors of the Karhunen-Loeve transforms, where the number L, is 
determined in accordance with HVS analysis. 

39. A method as claimed in claim 37, wherein said digital information is embedded in 
the first Li coefficients of the transformed image blocks, where the number L, is 
determined in accordance with HVS analysis. 

40. A method as claimed in claim 37, wherein said digital information is a noise-like 
watermark. 

41 . A method as claimed in claim 37, wherein the selecting step is determined in 
accordance with HVS analysis and a user's selection. 



30 
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42. A method of embedding digital information in a plurality of images, wherein the 
images are stored and classified into one or more classes and said method includes the 
following steps: 

generating one or more transforms from the images of said respective one or more 
5 classes, wherein said transforms are dependent on the images; 

generating transformed images by applying the one or more transforms to the 
images of said respective one or more classes; 

embedding the digital information in the transformed images; and 
generating the images data embedded with the digital information by applying 
10 one or more inverse transforms of the one or more transforms to the embedded 
transformed images. 

43 . A method of embedding digital information in a plurality of images, wherein the 
images are stored and classified into one or more classes and said method includes the 

15 following steps: 

generating one or more transforms from the images of said respective one or more 
classes, wherein said transforms are dependent on the images; 

generating transformed images by applying the one or more transforms to the 
images of said respective one or more classes; 
20 embedding the digital information in the one or more transforms or their inverses; 

and 

generating the images embedded with the digital information by applying 
one or more embedded inverse transforms to the transformed images. 

25 44. A method of embedding digital information into video images including a plurality 
of frames, wherein said method includes the following steps: 
extracting one or more key frames from the video images, 
generating one or more transforms from the key frames or a part thereof, wherein 
said transforms are dependent on the key frames or the part thereof; 
30 generating transformed key frames or part thereof by applying the one or more 

transforms to the key frames or the part thereof; 
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embedding the digital information in the transformed key frames or part thereof; 

and 

generating the key frames or part thereof embedded with the digital information by 
applying one or more inverse transforms of the one or more transforms to the embedded 
5 key frames or part thereof. 



45. A method of embedding digital information into video images including a plurality 
of frames, wherein said method includes the following steps: 

extracting one or more key frames from the video images; 
10 generating one or more transforms from the key frames or a part thereof, wherein 

said transforms are dependent on the key frames or the part thereof; 

generating transformed key frames or part thereof by applying the one or more 
transforms to the key frames or the part thereof; 

embedding the digital information in the one or more transforms or their inverses; 

15 and 

generating the key frames or part thereof embedded with the digital information by 
applying one or more embedded inverse transforms to the transformed key frames or 
part thereof. 

20 46. A method of registering an image, wherein the method includes the following steps: 
extracting key points from the image and their invariant attributes, 
extracting other global information from the image; 

aligning the extracted key points and their invariant attributes with the global 
information; and 

25 encrypting the aligned information to provide an image signature. 



47. A method of modifying a first image in accordance with an image signature, 
wherein the image signature includes encrypted key points of a second image and their 
invariant attributes and other aligned global information of the second image and the 
30 method includes the steps of: 
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decrypting the image signature to obtain the key points and their invariant attributes and 
other global information of the second image; 
extracting key points and their invariant attributes from the first image; 
extracting other global information from the first image; 
5 matching the key points and their invariant attributes and other global information of the 
first and second images; and 

normalising the first image based on said matched key points and their invariant 
attributes and other global information. 

10 

48. A method of extracting digital information from first digital multimedia data having 
embedded therein said digital information, wherein said method includes the following 
steps: 

deriving one or more embedded transforms from a reference file, wherein said 
1 5 embedded transforms are dependent on second digital multimedia data or a portion 
thereof; 

deriving transformed digital multimedia data from the reference file, wherein said 
transformed digital multimedia data is dependent on the second digital multimedia data 
or a portion thereof; 

20 generating transformed digital multimedia data by applying one or more of the 

derived embedded transforms to the first digital multimedia data or a portion thereof; 
and 

correlating the generated transformed digital multimedia with the derived 
transformed digital multimedia to check whether the digital information embedded in the 
25 first multimedia data is the same as that derived from the reference file. 

49. A method as claimed in claim 48, wherein said first digital multimedia data is an 
image. 

30 50. A method as claimed in claim 48, wherein said transform is a Karhunen-Loeve. 
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51. A method as claimed in claim 48, wherein said first and second digital 
multimedia data are images and said one or more transforms are Karhunen-Loeve 
transforms and said digital information is embedded in the first L, eigenvectors of the 
Karhunen-Loeve transforms, where the number L, is determined in accordance with 

5 HVS analysis. 

52. A method as claimed in claim 48, wherein said first and second digital multimedia 
data are first and second images respectively and the method includes the following 
steps; 

10 matching parameters of the first image with parameters of the second image stored 

in a reference file; and 

normalising said first image in accordance with said matched parameters. 

53. A method of extracting digital information from first digital multimedia data having 
15 embedded therein said digital information, wherein said method includes the following 

steps: 

deriving one or more transforms from a reference file, wherein said transforms are 
dependent on second digital multimedia data or a portion thereof; 

deriving transformed digital multimedia data from the reference file, wherein said 
20 transformed digital multimedia data is dependent on the second digital multimedia data 
or a portion thereof; 

generating embedded transformed digital multimedia data by applying one or more 
of the derived embedded transforms to the first digital multimedia data or a portion 
thereof; and 

25 correlating the generated embedded transformed digital multimedia with the 

derived transformed digital multimedia so as to extract the digital information. 

54. A method as claimed in claim 53, wherein said first digital multimedia data is an 
image. 

30 

55. A method as claimed in claim 53, wherein said transform is a Karhunen-Loeve. 
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56. A method as claimed in claim 53, wherein said digital multimedia data is an image 
and said one or more transforms are Karhunen-Loeve transforms and said digital 
information is embedded in the first L- t coefficients of the transformed digital 

5 multimedia data, where the number L, is determined in accordance with HVS 
analysis. 

57. A method as claimed in claim 53, wherein said first and second digital multimedia 
data are first and second images respectively and the method includes the following 

io steps; 

matching parameters of the first image with parameters of the second image stored 
in a reference file; and 

normalising said first image in accordance with said matched parameters. 

15 58. A method of embedding digital information in a digital image, wherein said method 
includes the following steps: 

partitioning the image into blocks of pixels; 

clustering the partitioned blocks into classes according to a homogeneity criteria; 
selecting one or more of the clusters in accordance with HVS analysis; 
20 averaging, for each selected cluster, the blocks of pixels; 

embedding, for each selected cluster, the digital information in the averaged 
blocks; and 

replacing, for each selected cluster, the blocks with the embedded averaged block 
for that cluster. 

25 

59. A method of extracting digital information from an image having embedded therein 
said digital information, wherein said method includes; 
partitioning the image into blocks of pixels; 

clustering the partitioned blocks into classes according to a homogeneity criteria; 
30 selecting one or more of the clusters in accordance with information obtained from 

a reference file; 
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retrieving, for each selected cluster, an average block of pixels stored in the 
reference file, 

averaging, for each selected cluster, the blocks of pixels of the image; and 
correlating, for each selected pixel, the averaged blocks of the image with the 
5 retrieved blocks to determine whether a watermark exists. 

60. An apparatus for embedding digital information in digital multimedia data, wherein 
said apparatus includes: 

means for generating one or more transforms from the digital multimedia data or a 
10 portion of the digital multimedia data, wherein said transforms are dependent on the 

digital multimedia data or the portion of the digital multimedia data; 

means for generating transformed digital multimedia data by applying the one or more 

transforms to the digital multimedia data or the portion of the digital multimedia data; 

means for embedding the digital information in the transformed digital multimedia data; 
15 and means for generating the digital multimedia data embedded with the digital 

information by applying one or more inverse transforms of the one or more transforms to 

the embedded transformed digital multimedia data. 

61. An apparatus for embedding digital information in digital multimedia data, wherein 
20 said apparatus includes: 

means for generating one or more transforms from the digital multimedia data or a 
portion of the digital multimedia data, wherein said transforms are dependent on the 
digital multimedia data or the portion of the digital multimedia data; 
means for generating transformed digital multimedia data by applying the one or more 
25 transforms to the digital multimedia data or the portion of the digital multimedia data; 
means for embedding the digital information in the one or more transforms or their 
inverses; and means for generating the digital multimedia data embedded with the digital 
information by applying one or more embedded inverse transforms to the transformed 
digital multimedia data. 
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62. An apparatus for embedding digital information in a digital image, wherein said 
apparatus includes: 

means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
5 criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis; 

means for generating an associated Karhunen-Loeve (K-L) transform for each 
selected cluster, wherein the image blocks of each selected cluster are used as a vector 
population for the purposes of generating the associated K-L transform; 
10 means for applying the associated K-L transform to the image blocks of the 

selected cluster to produce corresponding transformed image blocks; 

means for embedding, for each selected cluster, the digital information in the 
associated K-L transform; and 

means for applying, for each selected cluster, an inverse transform of the 
is embedded associated K-L transform to the transformed image blocks of the selected 
cluster. 

63. An apparatus for embedding digital information in a digital image, wherein said 
apparatus includes: 
20 means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 

criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis; 
means for generating an associated Karhunen-Loeve (K-L) transform for each 
25 selected cluster, wherein the image blocks of each selected cluster are used as a vector 
population for the purposes of generating the associated K-L transform; 

means for applying the associated K-L transform to the image blocks of the 
selected cluster to produce corresponding transformed image blocks; 

means for embedding, for each selected cluster, the digital information in the 
30 selected transformed image blocks; and 
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means for applying, for each selected cluster, an inverse transform of the 
associated K-L transform to the embedded transform image blocks of the selected 
cluster. 



5 64. An apparatus for embedding digital information in a digital image, wherein said 
apparatus includes: 

means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
criteria; 

10 means for selecting one or more of the clusters in accordance with HVS analysis; 

means for generating an associated Karhunen-Loeve (K-L) transform for each 
selected cluster, wherein the image blocks of each selected cluster are used as a vector 
population for the purposes of generating the associated K-L transform; 

means for applying the associated K-L transform to the image blocks of the 
15 selected cluster to produce corresponding transformed image blocks; 

means for selecting, for each selected cluster, either the associated K-L transform 
or its corresponding transformed image blocks for embedding; 

means for embedding, for each selected cluster, the digital information in the 
selected K-L transform or the selected transformed image blocks; 
20 means for applying, for those selected clusters in which the digital information has 

been embedded in the associated K-L transform, an inverse transform of the embedded 
associated K-L transform to the transformed image blocks of the selected cluster; and 

means for applying, for those selected clusters in which the digital information has 
been embedded in the transformed image blocks, an inverse transform of the associated 
25 K-L transform to the embedded transform image blocks of the selected cluster. 



65. An apparatus for embedding digital information in a plurality of images, wherein 
the images are stored and classified into one or more classes and said apparatus includes: 
means for generating one or more transforms from the images of said respective 
30 one or more classes, wherein said transforms are dependent on the images; 



WO 00/39954 



PCT/SG98/00110 



- 40- 

means for generating transformed images by applying the one or more transforms 
to the images of said respective one or more classes; 

means for embedding the digital information in the one or more transforms or their 
inverses, and 

5 means for generating the images embedded with the digital information by applying 

one or more embedded inverse transforms to the transformed images-. 

66. An apparatus for embedding digital information into video images including a 
plurality of frames, wherein said apparatus includes: 

io means for extracting one or more key frames from the video images; 

means for generating one or more transforms from the key frames or a part 
thereof, wherein said transforms are dependent on the key frames or the part thereof; 

means for generating transformed key frames or part thereof by applying the one 
or more transforms to the key frames or the part thereof, 
15 means for embedding the digital information in the transformed key frames or part 

thereof; and 

means for generating the key frames or part thereof embedded with the digital 
information by applying one or more inverse transforms of the one or more transforms to 
the embedded key frames or pan thereof 

20 

67. An apparatus for embedding digital information into video images including a 
plurality of frames, wherein said apparatus includes: 

means for extracting one or more key frames from the video images; 

means for generating one or more transforms from the key frames or a part 
25 thereof, wherein said transforms are dependent on the key frames or the part thereof; 

means for generating transformed key frames or part thereof by applying the one 
or more transforms to the key frames or the part thereof; 

means for embedding the digital information in the one or more transforms or their 
inverses; and 
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means for generating the key frames or part thereof embedded with the digital 
information by applying one or more embedded inverse transforms to the transformed 
key frames or part thereof. 



5 68. An apparatus for registering an image, wherein the apparatus includes: 
means for extracting key points from the image and their invariant attributes; 
means for extracting other global information from the image; 

means for aligning the extracted key points and their invariant attributes with the global 
information; and 

io means for encrypting the aligned information to provide an image signature. 

69. An apparatus for modifying a first image in accordance with an image signature, 

wherein the image signature includes encrypted key points of a second image and their 

invariant attributes and other aligned global information of the second image and the 
1 5 apparatus includes: 

means for decrypting the image signature to obtain the key points and their invariant 

attributes and other global information of the second image; 

means for extracting key points and their invariant attributes from the first image; 

means for extracting other global information from the first image; 
20 means for matching the key points and their invariant attributes and other global 

information of the first and second images; and 

means for normalising the first image based on said matched key points and their 
invariant attributes and other global information. 



25 70. An apparatus for extracting digital information from first digital multimedia data 
having embedded therein said digital information, wherein said apparatus includes: 

means for deriving one or more embedded transforms from a reference file, 
wherein said embedded transforms are dependent on second digital multimedia data or a 
portion thereof; 
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means for deriving transformed digital multimedia data from the reference file, 
wherein said transformed digital multimedia data is dependent on the second digital 
multimedia data or a portion thereof; 

means for generating transformed digital multimedia data by applying one or more 
5 of the derived embedded transforms to the first digital multimedia data or a portion 
thereof; and 

means for correlating the generated transformed digital multimedia with the 
derived transformed digital multimedia to check whether the digital information 
embedded in the first multimedia data is the same as that derived from the reference file. 

10 

71 . An apparatus for extracting digital information from first digital multimedia data 
having embedded therein said digital information, wherein said apparatus includes: 

means for deriving one or more transforms from a reference file, wherein said 
transforms are dependent on second digital multimedia data or a portion thereof; 
15 means for deriving transformed digital multimedia data from the reference file, 

wherein said transformed digital multimedia data is dependent on the second digital 
multimedia data or a portion thereof, 

means for generating embedded transformed digital multimedia data by applying 
one or more of the derived embedded transforms to the first digital multimedia data or a 
20 portion thereof; and 

means for correlating the generated embedded transformed digital multimedia with 
the derived transformed digital multimedia so as to extract the digital information. 

72. An apparatus for embedding digital information in a digital image, wherein said 
25 apparatus includes: 

means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis; 
30 means for averaging, for each selected cluster, the blocks of pixels; 
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means for embedding, for each selected cluster, the digital information in the 
averaged blocks; and 

means for replacing, for each selected cluster, the blocks with the embedded 
averaged block for that cluster. 

5 

73. An apparatus for extracting digital information from an image having embedded 
therein said digital information, wherein said apparatus includes; 
means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
io criteria; 

means for selecting one or more of the clusters in accordance with information 
obtained from a reference file; 

means for retrieving, for each selected cluster, an average block of pixels stored in 
the reference file; 

15 means for averaging, for each selected cluster, the blocks of pixels of the image; 

and 

means for correlating, for each selected pixel, the averaged blocks of the image 
with the retrieved blocks to determine whether a watermark exists. 

20 74. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in digital 
multimedia data, wherein said computer program product includes: 
means for generating one or more transforms from the digital multimedia data or a 
portion of the digital multimedia data, wherein said transforms are dependent on the 

25 digital multimedia data or the portion of the digital multimedia data; 

means for generating transformed digital multimedia data by applying the one or more 
transforms to the digital multimedia data or the portion of the digital multimedia data; 
means for embedding the digital information in the transformed digital multimedia data; 
and means for generating the digital multimedia data embedded with the digital 

30 information by applying one or more inverse transforms of the one or more transforms to 
the embedded transformed digital multimedia data. 
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75. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in digital 
multimedia data, wherein said computer program product includes: 

5 means for generating one or more transforms from the digital multimedia data or a 
portion of the digital multimedia data, wherein said transforms are dependent on the 
digital multimedia data or the portion of the digital multimedia data, 
means for generating transformed digital multimedia data by applying the one or more 
transforms to the digital multimedia data or the portion of the digital multimedia data; 

10 means for embedding the digital information in the one or more transforms or their 

inverses; and means for generating the digital multimedia data embedded with the digital 
information by applying one or more embedded inverse transforms to the transformed 
digital multimedia data. 

15 76. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in a digital 
image, wherein said computer program product includes: 
means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
20 criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis; 

means for generating an associated Karhunen-Loeve (K-L) transform for each 
selected cluster, wherein the image blocks of each selected cluster are used as a vector 
population for the purposes of generating the associated K-L transform; 
25 means for applying the associated K-L transform to the image blocks of the 

selected cluster to produce corresponding transformed image blocks; 

means for embedding, for each selected cluster, the digital information in the 
associated K-L transform; and 

means for applying, for each selected cluster, an inverse transform of the 
30 embedded associated K-L transform to the transformed image blocks of the selected 
cluster. 
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77. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in a digital 
image, wherein said computer program product includes: 

5 means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis; 
means for generating an associated Karhunen-Loeve (K-L) transform for each 
10 selected cluster, wherein the image blocks of each selected cluster are used as a vector 
population for the purposes of generating the associated K-L transform; 

means for applying the associated K-L transform to the image blocks of the 
selected cluster to produce corresponding transformed image blocks; 

means for embedding, for each selected cluster, the digital information in the 
15 selected transformed image blocks; and 

means for applying, for each selected cluster, an inverse transform of the 
associated K-L transform to the embedded transform image blocks of the selected 
cluster. 

78. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in a digital 
image, wherein said computer program product includes: 

means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 
criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis; 
means for generating an associated Karhunen-Loeve (K-L) transform for each 
selected cluster, wherein the image blocks of each selected cluster are used as a vector 
population for the purposes of generating the associated K-L transform; 

means for applying the associated K-L transform to the image blocks of the 
selected cluster to produce corresponding transformed image blocks; 



25 
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means for selecting, for each selected cluster, either the associated K-L transform 
or its corresponding transformed image blocks for embedding; 

means for embedding, for each selected cluster, the digital information in the 
selected K-L transform or the selected transformed image blocks; 
5 means for applying, for those selected clusters in which the digital information has 

been embedded in the associated K-L transform, an inverse transform of the embedded 
associated K-L transform to the transformed image blocks of the selected cluster; and 

means for applying, for those selected clusters in which the digital information has 
been embedded in the transformed image blocks, an inverse transform of the associated 
io K-L transform to the embedded transform image blocks of the selected cluster. 

79. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in a plurality of 
images, wherein the images are stored and classified into one or more classes and said 

1 5 computer program product includes: 

means for generating one or more transforms from the images of said respective 
one or more classes, wherein said transforms are dependent on the images; 

means for generating transformed images by applying the one or more transforms 
to the images of said respective one or more classes; 

20 means for embedding the digital information in the one or more transforms or their 

inverses; and 

means for generating the images embedded with the digital information by applying 
one or more embedded inverse transforms to the transformed images. 

25 80. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information into video 
images including a plurality of frames, wherein said computer program product includes: 
means for extracting one or more key frames from the video images; 
means for generating one or more transforms from the key frames or a part 
30 thereof, wherein said transforms are dependent on the key frames or the part thereof; 
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means for generating transformed key frames or part thereof by applying the one 
or more transforms to the key frames or the part thereof; 

means for embedding the digital information in the transformed key frames or part 
thereof; and 

5 means for generating the key frames or part thereof embedded with the digital 

information by applying one or more inverse transforms of the one or more transforms to 
the embedded key frames or part thereof. 



81. A computer program product including a computer readable medium having 
10 recorded thereon a computer program for embedding digital information into video 

images including a plurality of frames, wherein said computer program product includes: 

means for extracting one or more key frames from the video images; 

means for generating one or more transforms from the key frames or a part 
thereof, wherein said transforms are dependent on the key frames or the part thereof, 
15 means for generating transformed key frames or part thereof by applying the one 

or more transforms to the key frames or the part thereof; 

means for embedding the digital information in the one or more transforms or their 
inverses; and 

means for generating the key frames or part thereof embedded with the digital 
20 information by applying one or more embedded inverse transforms to the transformed 
key frames or part thereof 



82. A computer program product including a computer readable medium having 
recorded thereon a computer program for registering an image, wherein the computer 
25 program product includes: 

means for extracting key points from the image and their invariant attributes; 
means for extracting other global information from the image; 

means for aligning the extracted key points and their invariant attributes with the global 
information; and 

30 means for encrypting the aligned information to provide an image signature. 
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83. A computer program product including a computer readable medium having 
recorded thereon a computer program for modifying a first image in accordance with an 
image signature, wherein the image signature includes encrypted key points of a second 
image and their invariant attributes and other aligned global information of the second 

5 image and the computer program product includes: 

means for decrypting the image signature to obtain the key points and their invariant 
attributes and other global information of the second image; 
means for extracting key points and their invariant attributes from the first image; 
means for extracting other global information from the first image; 

io means for matching the key points and their invariant attributes and other global 
information of the first and second images; and 

means for normalising the first image based on said matched key points and their 
invariant attributes and other global information. 



15 84. A computer program product including a computer readable medium having 

recorded thereon a computer program for extracting digital information from first digital 
multimedia data having embedded therein said digital information, wherein said 
computer program product includes; 

means for deriving one or more embedded transforms from a reference file, 

20 wherein said embedded transforms are dependent on second digital multimedia data or a 
portion thereof; 

means for deriving transformed digital multimedia data from the reference file, 
wherein said transformed digital multimedia data is dependent on the second digital 
multimedia data or a portion thereof; 
25 means for generating transformed digital multimedia data by applying one or more 

of the derived embedded transforms to the first digital multimedia data or a portion 
thereof; and 

means for correlating the generated transformed digital multimedia with the 
derived transformed digital multimedia to check whether the digital information 
30 embedded in the first multimedia data is the same as that derived from the reference file. 
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85. A computer program product including a computer readable medium having 
recorded thereon a computer program for extracting digital information from first digital 
multimedia data having embedded therein said digital information, wherein said 
computer program product includes: 

5 means for deriving one or more transforms from a reference file, wherein said 

transforms are dependent on second digital multimedia data or a portion thereof, 

means for deriving transformed digital multimedia data from the reference file, 
wherein said transformed digital multimedia data is dependent on the second digital 
multimedia data or a portion thereof; 
io means for generating embedded transformed digital multimedia data by applying 

one or more of the derived embedded transforms to the first digital multimedia data or a 
portion thereof; and 

means for correlating the generated embedded transformed digital multimedia with 
the derived transformed digital multimedia so as to extract the digital information. 

15 

86. A computer program product including a computer readable medium having 
recorded thereon a computer program for embedding digital information in a digital 
image, wherein said computer program product includes: 

means for partitioning the image into blocks of pixels; 
20 means for clustering the partitioned blocks into classes according to a homogeneity 

criteria; 

means for selecting one or more of the clusters in accordance with HVS analysis, 
means for averaging, for each selected cluster, the blocks of pixels; 
means for embedding, for each selected cluster, the digital information in the 
25 averaged blocks; and 

means for replacing, for each selected cluster, the blocks with the embedded 
averaged block for that cluster. 

87. A computer program product including a computer readable medium having 

30 recorded thereon a computer program for extracting digital information from an image 
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having embedded therein said digital information, wherein said computer program 
product includes; 

means for partitioning the image into blocks of pixels; 

means for clustering the partitioned blocks into classes according to a homogeneity 



means for selecting one or more of the clusters in accordance with information 
obtained from a reference file; 

means for retrieving, for each selected cluster, an average block of pixels stored in 
the reference file; 

means for averaging, for each selected cluster, the blocks of pixels of the image; 

and 

means for correlating, for each selected pixel, the averaged blocks of the image 
with the retrieved blocks to determine whether a watermark exists. 



5 criteria; 



WO 00/39954 



PCT/SG98/00110 



.1/8 




START 



100 



1 102 

* — 



INPUT 



PREPROCESS 



104 



REFERENCE FILE 



122 



BLOCK AND CLUSTER 



106 



CLUSTER SELECTION 



108 



KLT TRANSFORM 



110 



DOMAIN SELECTION 



112 



WATERMARK EMBEDDING 



114 



INVERSE KLT TRANSFORM 



116 



OUTPUT 



118 



120 




FIG. t 



END 



SUBSTITUTE SHEET (RULE 26) 




FIG. 2 



SUBSTITUTE SHEET (RULE 26) 



WO 00/39954 



-3/8- 



PCT/SG98/001I0 



300 



INPUT IMAGE 



302 



PREPROCESSING 



IMAGE SUMMARIZING 



ATTRIBUTES 
CALCULATION AND 
ALIGNMENT 



308 



! ENCRYPTION 



310 



I 

ILE 

I 



312 



REFERENCE FILE FORMATION r 




314 



FIG. 3 



SUBSTITUTE SHEET (RULE 26) 



PCT/SG98/00110 

WO 00/39954 

-4/8- 



400 




WATERMARKED IMAGE 



402 



404 \ i r 

N IMAGE REGISTRATION \*~ 



REFERENCE 
FILE 



40 S I , l 

V^RECOVERING IMAGE \4 



403 



106. ^ 
^ BLOCK AND 



CLUSTER 



108 



^ | CLUSTER SELECTION ) «--; 



408 > I 

V.| DOMAIN SELECTION 



410. ^ 

^ | K-L TRANSFORM } +- 
412^ 




CORRELATION 



414. 




EXTRACTED 
WATERMARK 




FIG. 4 



SUBSTITUTE SHEET (RULE 26) 



WO 00/39954 PCT/SG98/001 1 0 

-5/8- 




500 



INPUT IMAGE 



502 



I 



PREPROCESSING 



504 



503 



REFERENCE FILE 



506 



CONTENT SUMMARIZATION 



RULE-BASED IMAGE 



510 



IMAGE RECOVERING 



512 



T 



FIG. 5 



SUBSTITUTE SHEET (RULE 26) 



WO 00/39954 



PCT/SG98/00110 



• 6/8- 



START 



600 



INPUT 



l/V 



102 



PREPROCESS 



104 



BLOCK AND CLUSTER 



106 



CLUSTER SELECTION 



108 



AVERAGING BLOCKS 



1/-V 



602 



WATERMARK EMBEDDING 



604 

r 7 



RECONSTRUCT IMAGE 



606 



608 
OUTPUT 



END 



610 



FIG. 6 



SUBSTITUTE SHEET (RULE 26) 



WO 00/39954 



PCT/SG98/00in> 



-7/8- 



700 



INPUT 



102 



714 



REFERENCE FILE 



PREPROCESS 



104 



I 



BLOCK AND CLUSTER 



106 



CLUSTER SELECTION 



704 



706 



OBTAIN AVERAGE BLOCK 



DETERMINE AVERAGE 
BLOCK OF TEST IMAGE 



708 



CORRELATE 



710 



FIG. 7 



SUBSTITUTE SHEET (RULE 26) 



WO 00/39954 



PCT/SG98/00110 



8/8 



800 



830 



VIDEO 
DISPLAY 



816 



COMMUNICATION 
CHANNEL 



10 812 



802 



VIDEO 
INTERFACE 



STORAGE 
DEVICE 



I 



808A 



I/O 

INTERFACE 



5 



804 



81 



~7 

806 





k * 

r 814 


i 




X 


PROCESSOR 




MEMORY 




I/O 

INTERFACE 



3 

808B 



KEYBOARD 



820 



A' 



FIG. 8 



SUBSTITUTE SHEET (RULE 26) 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/SG 98/00110 



A. CLASSIFICATION OF SUBJECT MATTER 
IPC 7 : H04L 9/00, G06T 1 1/00 

According t o International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC 7 : H04L, G06T, H04N 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 
WPI 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



EP 0859337 A2 (NEC Corporation) 19 August 1998 (19.08.98), 
abstract; column 2, line 2 to column 5, line 55; column 7, line 52 to 
column 9, line 12; figl. 1, 2, 10, 1 1. 



US 5809139 A ( Girod et ail.) 15 September 1998 (15.09.98), abstract; 
column 1, line 8 to column 3, line 15; column 3, line 47 to column 5, 
line 21; fig. 1. 



US 4910716 A (Kirlin et al.) 20 March 1990 (20.03.90), column 1, 
line 47 to column 2, line 28; column 2, line 60 to column 4, line 66. 



US 5657390 A (Elgamal et al.) 12 August 1997 (12.08.97), abstract; 
claim 1 . 



1,2,7,12,15,16, 
21,26,29,31,33, 
37,40,42-49,53, 
58-87 

1,2,7,12,15,16, 
21,26,29,31,33, 
37,40,42-49,53, 
58-87 

5,203,33,37, 
50,55,62-64, 
76-78 

74-87 



□ 

Further documents are listed in the continuation of Box C. 



See patent family annex. 



* Special categories of cited documents: 

„A" document defining the general state of the art which is not 

considered to be of particular relevance 
„E" earlier application or patent but published on or after the international 

filing date 

M L" document which may throw doubts on priority claim(s) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified) 

„0" document referring to an oral disclosure, use, exhibition or other 
means 

„F* document published prior to the international filing date but later than 
the priority date claimed 



,,'P* later document published after the international filing date or priority 

date and not in conflict with the application but cited to understand 

the principle or theory underlying the invention 
„X" document of particular relevance; the claimed invention cannot be 

considered novel or cannot be considered to involve an inventive step 

when the document is taken alone 
v.Y" document of particular relevance; the claimed invention cannot be 

considered to involve an inventive step when the document is 

combined with one or more other such documents, such combination 

being obvious to a person skilled in the art 
„&** document member of the same patent family 



Date of the actual completion of the international search 

13 January 2000 (13.01.00) 



Date of mailing of the international search report 

04 February 2000 (04.02.00) 



Name and mailing adress of the ISA/AT 

Austrian Patent Office 
Kohlmarkt 8-10; A- 101 4 Vienna 

Facsimile No. 1/53424/200 



Form PCT/ISA/2 10 (second sheet) (July 1998) 



Authorized officer 

Hajos 

Telephone No. 1/53424/410 



♦ 



INTERNATIONAL SEARCH REPORT 

Information on patent family members 



International aDDlication No. 

PCT/SG 98/00110 



Patent document cited 
in search report 



EP 
EP 



A2 
A3 



859337 
859337 



5809139 



OS 



4910716 



US 



5657360 



Publication 
date 



Patent family 
member(s) 



Publication 
date 



19-08-1998 
27-01-1999 



CA 
JP 



AA 
A2 



2227381 
10313402 



14-08-1998 
24-11-1998 



15-09-1998 



20-03-1990 



12-08-1997 



JP 



A2 8146184 



07-06-1996 



PCT/ISA/210 (patent family annex) fluly 1998) 



